Other Subreddit’s Data
Riccardo Ferrarese
19/3/2021
Questo report contiene l’analisi di base riguardo gli altri subreddit scaricati.
##
## Attaching package: 'lubridate'## The following objects are masked from 'package:base':
##
## date, intersect, setdiff, union##
## Attaching package: 'dplyr'## The following objects are masked from 'package:stats':
##
## filter, lag## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, union## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.1 ──## ✓ ggplot2 3.3.3 ✓ purrr 0.3.4
## ✓ tibble 3.1.1 ✓ stringr 1.4.0
## ✓ readr 1.4.0 ✓ forcats 0.5.1## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## x lubridate::as.difftime() masks base::as.difftime()
## x lubridate::date() masks base::date()
## x dplyr::filter() masks stats::filter()
## x lubridate::intersect() masks base::intersect()
## x dplyr::lag() masks stats::lag()
## x lubridate::setdiff() masks base::setdiff()
## x lubridate::union() masks base::union()## Registered S3 method overwritten by 'sets':
## method from
## print.element ggplot2##
## Attaching package: 'sets'## The following object is masked from 'package:forcats':
##
## %>%## The following object is masked from 'package:stringr':
##
## %>%## The following object is masked from 'package:purrr':
##
## %>%## The following object is masked from 'package:tibble':
##
## %>%## The following object is masked from 'package:tidyr':
##
## %>%## The following object is masked from 'package:dplyr':
##
## %>%## The following objects are masked from 'package:lubridate':
##
## as.interval, interval, is.interval##
## Attaching package: 'tidytable'## The following object is masked from 'package:sets':
##
## %>%## The following object is masked from 'package:stats':
##
## dt##
## Attaching package: 'igraph'## The following object is masked from 'package:sets':
##
## %>%## The following objects are masked from 'package:purrr':
##
## compose, simplify## The following object is masked from 'package:tibble':
##
## as_data_frame## The following object is masked from 'package:tidyr':
##
## crossing## The following objects are masked from 'package:dplyr':
##
## as_data_frame, groups, union## The following objects are masked from 'package:lubridate':
##
## %--%, union## The following objects are masked from 'package:stats':
##
## decompose, spectrum## The following object is masked from 'package:base':
##
## union##
## Attaching package: 'tidygraph'## The following object is masked from 'package:igraph':
##
## groups## The following object is masked from 'package:sets':
##
## %>%## The following object is masked from 'package:stats':
##
## filter## Package version: 3.0.0
## Unicode version: 13.0
## ICU version: 66.1## Parallel computing: 16 of 16 threads used.## See https://quanteda.io for tutorials and examples.##
## Attaching package: 'quanteda'## The following object is masked from 'package:tidygraph':
##
## convert## The following object is masked from 'package:sets':
##
## %>%##
## Attaching package: 'quanteda.textplots'## The following object is masked from 'package:tidygraph':
##
## as.igraph## The following object is masked from 'package:igraph':
##
## as.igraph##
## Attaching package: 'stringdist'## The following object is masked from 'package:tidyr':
##
## extract## Loading required package: koRpus.lang.en## Loading required package: koRpus## Loading required package: sylly## For information on available language packages for 'koRpus', run
##
## available.koRpus.lang()
##
## and see ?install.koRpus.lang()##
## Attaching package: 'koRpus'## The following objects are masked from 'package:quanteda':
##
## tokens, types## The following object is masked from 'package:readr':
##
## tokenize##
## Attaching package: 'hunspell'## The following object is masked from 'package:quanteda':
##
## dictionaryPANCAKE SWAP
Come prima cosa scarichiamo i dati riguardanti il subreddit da analizzare.
save <- FALSE
dataset <- "pancakeswap"
com_raw_data <- read.csv("../Data/src/pancakeswap_com.csv")
# create and save data (df + corpus)
data = df.create(com_raw_data, "../Data/", dataset, filter_post = 0)
remove(doge_com_raw_data)Facciamo un summary dei dataframe per venderne il contenuto:
if (save) {
data <- readRDS(paste0("../Data/", dataset, ".rds"))
}
print_("REDDIT's COMMENT DF: ")##
## REDDIT's COMMENT DF:data$df_comm %>%
select(id, date, author) %>%
head(3)print_("REDDIT's POST DF: ")##
## REDDIT's POST DF:summary(data$df_post)## postID body n_com
## t3_j1d97y: 1 Length:2411 Min. : 1.000
## t3_jxqv4d: 1 Class :character 1st Qu.: 2.000
## t3_kkjvfx: 1 Mode :character Median : 4.000
## t3_kkkr85: 1 Mean : 6.667
## t3_kl7hj5: 1 3rd Qu.: 8.000
## t3_komqkz: 1 Max. :109.000
## (Other) :2405Ci rendiamo conto della differenza di commenti raccolti guardando la dimensione del campo body. Si hanno infatti 2411 commenti corrispondenti a 2405 post.
Si va poi ad analizzare i token e bigrammi estratti dal corpus di testo.
if (save) {
data <- readRDS(paste0("../Data/", dataset, ".rds"))
}
com_corpus <- data$corpus_comm
remove(data)
# remove stopwords!!
tidy <- corpus.tokenize_dfmTidy(com_corpus, spell_checking = TRUE,
mode_correction = 0)
clean_tidy <- corpus.clean_tidy(tidy$tidy, mode = "stem")
word_counts <- corpus.countPlot_tidy(clean_tidy, threshold_count = 50)
remove(clean_tidy)set.seed(190)
dff <- tidy$dfm
# colour of words from least to most frequent
textplot_wordcloud(dff, min_count = 200, color = c("pink", "red",
"green3", "purple", "orange", "blue3"))
remove(dff)A differenza della wordcloud fatta per il subreddit di Dogecoin possiamo notare come non si hanno più parole legate al solo tema di comprare/vendere cryptovalute, ma sono presenti anche parole riguardanti diversi wallet probabilmente dovute a consigli o discussioni su quali utilizzare.
words.classSentiment(word_counts)
Troviamo anche una differenza minore tra sentimenti positivi e negativi, ma rimane comunque bilanciato a causa delle incertezze dovute all’investire in un mercato con molta volatilità . Questo può esser confermato anche dalla classificazione dei termini sottostante.
word_sentiment <- words.computeSentiment(word_counts, n_filter = 10)
Si va quindi ad analizzare i bigrammi per vedere se le ipotesi fatte sopra sia rispettate anche tenendo conto delle parole adiacenti.
tidy_bigrams <- corpus.tokenize_dfmTidy(com_corpus, dfm_b = FALSE,
spell_checking = FALSE, ngrams = 2)
clean_tidy_bigrams <- corpus.clean_tidy(tidy_bigrams$tidy, ngrams = 2)
remove(tidy_bigrams)
bigrams_counts <- corpus.countPlot_tidy(clean_tidy_bigrams, threshold_count = 25,
ngrams = 2)
Analizzando i bigrammi si può notare ancora la notevole differenza con i bigrammi estratti dal corpus relativo a Dogecoin. Infatti non sono più presenti coppie di parole che incitano all’acquisto tra quelle con maggior frequenza, ma sembrerebbe che in questo subreddit ci siano anche discussioni sui temi generali delle cryptovalute, per esempio liquidity pool, seed phrase e smart chain. Questo può star a indicare che nella comunità di PancakeSwap ci siano persone un po’ meno esperte che cercano consigli o espongono dubbi sul mondo delle cryptovalute, bisognerebbe però approfondire meglio il contesto di ciascun bigramma.
bigrams_class_sentiment <- words.classSentiment(bigrams_counts,
n_filter_sentiment = 200, ngrams = 2)
bigrams_sentiment <- words.computeSentiment(bigrams_counts, n_filter = 2,
ngrams = 2)
# saveRDS(bigrams_sentiment, paste0('../Data/', dataset,
# '_bigrams_sentiment.rds')Si hanno anche un numero molto minore di parole considerate negativa, teniamo però conto che esiste un enorme differenza tra il numero di dati raccolti per Dogecoin rispetto a quelli per PancakeSwap.
words.network(bigrams_counts, n_filter = 30)
remove(tidy_bigrams, clean_tidy_bigrams)
remove(bigrams_counts)
remove(bigrams_class_sentiment)
remove(bigrams_sentiment)BITCOIN - SubReddit
Come prima cosa scarichiamo i dati riguardanti il subreddit da analizzare. Nel Dataset riguardante il subreddit di Reddit abbiamo un milione e seicento mila osservazioni, per limitare l’uso di memoria si va ad analizzare solo i commenti in merito all’anno 2021.
save = FALSE
dataset = "bitcoin"
com_raw_data <- read.csv("../Data/src/Bitcoin_com.csv")
# create and save data (df + corpus)
data = df.create(com_raw_data, "../Data/", dataset, filter_post = 0)
remove(com_raw_data)Facciamo un summary dei dataframe per venderne il contenuto:
if (save) {
data <- readRDS(paste0("../Data/", dataset, ".rds"))
}
print_("REDDIT's COMMENT DF: ")##
## REDDIT's COMMENT DF:data$df_comm %>%
select(id, date, author) %>%
head(3)print_("REDDIT's POST DF: ")##
## REDDIT's POST DF:summary(data$df_post)## postID body n_com
## : 1 Length:94665 Min. : 1.00
## Bitcoin : 1 Class :character 1st Qu.: 2.00
## t3_cd1fr8: 1 Mode :character Median : 6.00
## t3_cd5p4x: 1 Mean : 16.81
## t3_cddheu: 1 3rd Qu.: 13.00
## t3_cdkxbx: 1 Max. :4534.00
## (Other) :94659if (save) {
data <- readRDS(paste0("../Data/", dataset, ".rds"))
}
com_corpus <- data$corpus_comm
remove(data)
docvars(com_corpus)# filter 2021 post with quanteda library
com_corpus_ <- corpus_subset(com_corpus, data > as.Date("2020-12-31"))
remove(com_corpus)
# remove stopwords!!
tidy <- corpus.tokenize_dfmTidy(com_corpus_)
clean_tidy <- corpus.clean_tidy(tidy$tidy, mode = "stem")
word_counts <- corpus.countPlot_tidy(clean_tidy, threshold_count = 5000)
remove(clean_tidy)set.seed(190)
dff <- tidy$dfm
# colour of words from least to most frequent
textplot_wordcloud(dff, min_count = 10000, color = c("grey",
"red", "green3", "purple", "orange", "blue3"))
remove(dff, tidy)Anche in questo caso si parla di wallet e non troviamo una grossa differenza come in dogecoin riguardo parole come ‘buy’ e ‘hold’ rispetto tutti gli altri termini.
words.classSentiment(word_counts)
word_sentiment <- words.computeSentiment(word_counts, n_filter = 10)
tidy_bigrams <- corpus.tokenize_dfmTidy(com_corpus_, dfm_b = FALSE,
spell_checking = FALSE, ngrams = 2)
remove(com_corpus_)
clean_tidy_bigrams <- corpus.clean_tidy(tidy_bigrams$tidy, ngrams = 2)
remove(tidy_bigrams)
bigrams_counts <- corpus.countPlot_tidy(clean_tidy_bigrams, threshold_freq = 1e-04,
threshold_count = 1000, ngrams = 2)
remove(clean_tidy_bigrams)Anche i bigrammi ci confermano che le discussioni in questo subreddit non sono solamente incentrate sull’acquisto della moneta, ma tra le più frequenti compaiono anche parole come private key, original article, hardware wallet.
bigrams_class_sentiment <- words.classSentiment(bigrams_counts,
n_filter_sentiment = 10000, ngrams = 2)
bigrams_sentiment <- words.computeSentiment(bigrams_counts, n_filter = 7,
ngrams = 2)
# saveRDS(bigrams_sentiment, paste0('../Data/', dataset,
# '_bigrams_sentiment.rds')words.network(bigrams_counts, n_filter = 2000)
remove(tidy_bigrams)
remove(bigrams_counts)
remove(bigrams_class_sentiment)
remove(bigrams_sentiment)ETHEREUM
Come prima cosa scarichiamo i dati riguardanti il subreddit da analizzare.
save = FALSE
dataset = "ethtrader"
com_raw_data <- read.csv("../Data/src/ethtrader_com.csv")
# create and save data (df + corpus)
data = df.create(com_raw_data, "../Data/", dataset, filter_post = 0)
remove(com_raw_data)Facciamo un summary dei dataframe per venderne il contenuto:
if (save) {
data <- readRDS(paste0("../Data/", dataset, ".rds"))
}
print_("REDDIT's COMMENT DF: ")##
## REDDIT's COMMENT DF:data$df_comm %>%
select(id, date, author) %>%
head(3)print_("REDDIT's POST DF: ")##
## REDDIT's POST DF:summary(data$df_post)## postID body n_com
## t3_chbx06: 1 Length:26142 Min. : 1.000
## t3_cp8c2w: 1 Class :character 1st Qu.: 1.000
## t3_crc2r9: 1 Mode :character Median : 3.000
## t3_crl0d2: 1 Mean : 9.109
## t3_cups8q: 1 3rd Qu.: 7.000
## t3_cvihol: 1 Max. :26179.000
## (Other) :26136if (save) {
data <- readRDS(paste0("../Data/", dataset, ".rds"))
}
com_corpus <- data$corpus_comm
remove(data)
# remove stopwords!!
tidy <- corpus.tokenize_dfmTidy(com_corpus, spell_checking = TRUE,
mode_correction = 0)
clean_tidy <- corpus.clean_tidy(tidy$tidy, mode = "stem")
word_counts <- corpus.countPlot_tidy(clean_tidy, threshold_count = 500)
remove(clean_tidy)set.seed(190)
dff <- tidy$dfm
# colour of words from least to most frequent
textplot_wordcloud(dff, min_count = 200, color = c("pink", "red",
"green3", "purple", "orange", "blue3"))
remove(dff, tidy)words.classSentiment(word_counts)
word_sentiment <- words.computeSentiment(word_counts, n_filter = 10)
tidy_bigrams <- corpus.tokenize_dfmTidy(com_corpus, dfm_b = FALSE,
spell_checking = FALSE, ngrams = 2)
clean_tidy_bigrams <- corpus.clean_tidy(tidy_bigrams$tidy, ngrams = 2)
remove(tidy_bigrams)
bigrams_counts <- corpus.countPlot_tidy(clean_tidy_bigrams, threshold_count = 300,
ngrams = 2)
remove(clean_tidy_bigrams)bigrams_class_sentiment <- words.classSentiment(bigrams_counts,
n_filter_sentiment = 3000, ngrams = 2)
bigrams_sentiment <- words.computeSentiment(bigrams_counts, n_filter = 4,
ngrams = 2)
# saveRDS(bigrams_sentiment, paste0('../Data/', dataset,
# '_bigrams_sentiment.rds')words.network(bigrams_counts, n_filter = 25)
remove(bigrams_counts)
remove(bigrams_class_sentiment)
remove(bigrams_sentiment)ELON MUSK
Questo dataset non fa parte della tematica del report, ma si è andato ad analizzarlo per curiosità visto che il termini ‘elon musk’ risulta molto frequente nel reddit di dogecoin. Come prima cosa scarichiamo i dati riguardanti il subreddit da analizzare.
save = FALSE
dataset = "elonmusk"
com_raw_data <- read.csv("../Data/src/elonmusk_com.csv")
# create and save data (df + corpus)
data = df.create(com_raw_data, "../Data/", dataset, filter_post = 0)
remove(com_raw_data)Facciamo un summary dei dataframe per venderne il contenuto:
if (save) {
data <- readRDS(paste0("../Data/", dataset, ".rds"))
}
print_("REDDIT's COMMENT DF: ")##
## REDDIT's COMMENT DF:data$df_comm %>%
select(id, date, author) %>%
head(3)print_("REDDIT's POST DF: ")##
## REDDIT's POST DF:summary(data$df_post)## postID body n_com
## t3_cnfdol: 1 Length:4903 Min. : 1.00
## t3_d25ver: 1 Class :character 1st Qu.: 1.00
## t3_docw7f: 1 Mode :character Median : 3.00
## t3_dprqr4: 1 Mean : 16.67
## t3_dpylfu: 1 3rd Qu.: 11.00
## t3_dqxrys: 1 Max. :857.00
## (Other) :4897if (save) {
data <- readRDS(paste0("../Data/", dataset, ".rds"))
}
com_corpus <- data$corpus_comm
remove(data)
# remove stopwords!!
tidy <- corpus.tokenize_dfmTidy(com_corpus, spell_checking = TRUE,
mode_correction = 0)
clean_tidy <- corpus.clean_tidy(tidy$tidy, mode = "stem")
word_counts <- corpus.countPlot_tidy(clean_tidy, threshold_count = 300)
remove(clean_tidy)set.seed(190)
dff <- tidy$dfm
# colour of words from least to most frequent
textplot_wordcloud(dff, min_count = 200, color = c("grey", "red",
"green3", "purple", "orange", "blue3"))
remove(dff)words.classSentiment(word_counts)
Un osservazione che si può fare è che la divisione nei sentimenti dei termini è più o meno sempre la stessa, una maggioranza di termini negativi seguiti da quelli positivi, con fiducia e rabbia che si contrappongono e una presenza non scontata di paura.
word_sentiment <- words.computeSentiment(word_counts, n_filter = 10)
tidy_bigrams <- corpus.tokenize_dfmTidy(com_corpus, dfm_b = FALSE,
spell_checking = FALSE, ngrams = 2)
clean_tidy_bigrams <- corpus.clean_tidy(tidy_bigrams$tidy, ngrams = 2)
remove(tidy_bigrams)
bigrams_counts <- corpus.countPlot_tidy(clean_tidy_bigrams, bool_plot_frequency = FALSE,
threshold_count = 75, ngrams = 2)
Analizzando i bigrammi si può notare ancor più maggiormente che la gran parte delle parole si riferiscono a spingere gli utenti ad acquistare e mantenere i dogecoin nel proprio wallet digitale.
bigrams_class_sentiment <- words.classSentiment(bigrams_counts,
n_filter_sentiment = 2000, ngrams = 2)
bigrams_sentiment <- words.computeSentiment(bigrams_counts, n_filter = 2,
ngrams = 2)
# saveRDS(bigrams_sentiment, paste0('../Data/', dataset,
# '_bigrams_sentiment.rds')Interessante notare come nelle parole classificate come negative sia presente il termine rischio, probabilmente inteso verso le azioni delle sue aziende o verso le sue idee visionarie.
words.network(bigrams_counts, n_filter = 150)
remove(tidy_bigrams)
remove(bigrams_counts)
remove(bigrams_class_sentiment)
remove(bigrams_sentiment)